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I. 


INTRODDCTICN 


When  is  a  program  considered  to  be  trivial?  One  answer 
to  this  question  heard  very  often  is  ’’When  it  contains  no 
bugs".  although  this  statement  might  be  questionable,  the 
converse  is  true,  as  there  are  few  nontrivial  programs  that 
do  not  certain  bugs.  As  the  author  of  a  critical  and 
fundamental  study  of  program  design  states;  "...These  tugs 
can  never  be  completely  exorcised  in  any  program  over  some 
critical  decree  of  complexity.  Six  months  or  even  seven 
years  after  'final  debugging'  errors  crop  up  inevitably  in 
the  best  of  prog  rajas 4  ].  This  is  a  fact  one  has  to  live 
with,  and  there  are  .only  two  things  one  can  do  about  it; 
First  to  reduce  the  possibilities  for  bugs  by  careful  design 
and  use  cf  modern  programming  techniques,  second  to  devise 
careful  testing  techniques  to  detect  and  locate  the  hugs 
still  remaining  in  tie  program. 

Fig.  1  shows  the  relationship  between  hardware  and 
software  cost  in  the  D.S.  during  the  period  from  1955  to 
19£5.  Cue  to  the  fact  that  the  software  cost  continues  to 
rise  and  that  about  50S  of  this  cost  is  for  testing  and 
integration  cf  a  system  [7],  it  is  important  to  obtain  a 
realistic  assessment  of  how  much  effort  has  to  be  spent  to 
test  the  newly  designed  program  based  on  its  sixe,  structure 
and  characteristics.  If  one  is  able  to  determine  ir  the 
design  stage  the  best  possible  structure  with  respect  to  the 
error  detection  capabilities,  then  bugs  can  be  avoided  and 
testing  will  be  reduced.  Also  early  in  the  development  cf  a 
project  a  realistic  allocation  of  coding  and  testing 
resources  could  be  made. 
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Ficure  1  -  SOFTtfABE  COST  TREND  IN  THE  D,S. 

[  Datamation,  Sept.  1974,  pg.  75] 

In  order  to  address  these  problems,  a  Software  Error 
Detection  Simulation  Model  has  been  developed  [7,10].  This 
model  was  was  used  to  identify  program  complexity  measures 
which  were  correlated  with  error  detection.  Naval  Tactical 
Data  System  picgrams  were  used  for  this  purpose. 

The  structures  of  these  NTDS-programs  have  been  analyzed 
(see  Chapter  ^I)  and  put  into  the  form  of  directed  graphs. 

The  date  gained  from  the  directed  graph  representation 
were  used  as  inputs  for  the  Error  Detection  Simulation 
Model.  The  results  gained  and  the  conclusions  and 

recommendations  draJin  from  these  results  are  shown  in 
Chapter  1/II .  For  reasons  of  security,  the  programs  or  the 
parts  of  them  are  not  identified  by  names.  Instead,  a 
sequential  cumber  scheme  for  identifying  the  programs  has 
been  employed. 
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This  wcrk  is  part  of  a  research  effort  sponsored  ty  the 
NACC  to  get  software  evaluation  aids  which  provide  an 
economical  assessment  of  the  design  and  testing  effort 
needed  for  the  development  of  avionics  and  other  complex 
software  projects. 


Because  it  is 
debugging  can  be 
technigues  in  the 
chapter  shows  the 
to  the  prchlem  of 


felt  that  efforts  in  testing  and  in 
more  successful  if  cne  employs  icdern 
production  of  programs,  an  introductory 
relevance  of  modern  programming  techniques 
program  testing  and  maintenance. 
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II .  DEFINITIONS 


There  was  originally  a  lack  of  commonly  used  definitions 
for  program  testing.  Only  recently  has  a  '’definitional 
framework"  eaerged  and  very  good  program  testing  definitions 
are  found  in  Eef.  8,  pg.  7  -  14.  In  order  to  be  consistent 
and  to  specify  the  meaning  of  keywords  within  this  thesis, 
the  fcllcwinc  definitions  have  been  adopted; 


1 .  Etc^raj  Structure 


The  structure  of  a  program  is  a  description  of  the 
underly,ing  logic  and  data  flow  as  represented  in  the 
form  cf  a  directed  graph  with  its  set  of  nodes  and 
edges  (arcs)  . 


2 •  Eeacbability  Index 


Beackatility  index  is 
possibilities  to  get  to  a 
over  all  nodes  of  the 
computed  with  the  formula 


a  measurement 
specified  node, 
directed  graph. 


o  f  the 
computed 
It  is 


R 


to  node 


(i)  • 


Debugging 


Debugging  is  the  action  one  takes  to  locate  and 
correct  a  known  or  detected  error  in  a  program. 
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4. 


lestic^ 

Testing  is  the  action  to  check  whether  a  picgram 
meets  its  specifications  and  to  establish  the 
presence  of  errors  in  it. 

5  .  li f €  cycle  of  a  program 

The  life  cycle  of  a  program  consists  of  the 
fcllcwing  phases: 

-  design 

-  Coding 

-  Cebucging 

-  Testing 

-  Eroduction  and  maintenance. 


III.  MO^EN  PBOGEAM^NG  TECHNIQUES 


Iwo  recent  developments  in  the  theory  and  practice  of 
software  cevelcpment  are  addressed  here  as  important  because 
they  are  relevant  not  only  for  the  actual  writing  cf  the 
code  of  the  program,  but  also  to  debugging,  testing,  and 
integrating  software  systems  as  well,  namely  the  advent  of 
modular  and  structured  programming.  The  advantages  of  these 
technigues  are  obvious  for  the  programmer  when  he  develops 
his  program,  programs  written  using  these  technigues  are 
easier  to  read  and  to  understand  as  far  as  the  flow  cf  the 
logic  is  concerned.  Also,  the  tester  can  better  understand 
the  logic  cf  a  program  when  these  technigues  are  employed, 
furthermore,  it  has  been  proposed  for  structured  programs  to 
eliminate  flowcharts  as  media  cf  communication  [13],  sc  it 
is  necessary  tc  undexstand  how  much  testing,  integration  and 
maintenance  cf  software  are  influenced  by  this  development. 


A.  flCDDlAB  ifCGBAKMING 


Modular  programming  is  a  system  to  develop  programs  as  a 
set  of  interrelated  individual  units  (called  modules)  which 
later  can  be  linked  together  tc  form  a  complete  program  [9]* 
Thus  modular  programming  is  not  simply  splitting  up  a 
program  intc  several  parts  (subroutines) ,  but  rather 
dividing  the  software  according  to  the  functions  tc  be 
performed.  The  designer  faces  the  one  crucial  problem  which 
will  determine  success  or  failure,  namely  to  specify 
completely  and  carefully  the  interface  between  the 
individual  mcdules. 
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Modules  as  individual  program  units  should  hav€  the 
following  properties: 

(1)  Cre  ircdule  should  perfcrir  only  one  basic  function 

(2)  The  size  of  a  module  should  be  such  that  it  is 
easily  understood  and  contains  crly  a  moderate 
ancurt  of  code 

(3)  module  should  be  designed  in  such  a  way  that  it 
has  only  a  few  control  or  data  paths 

(4)  Cne  nodule  should  process  only  a  small  amount  of 
data . 


The  design  of  programs  in  this  way  leads  not  only  to 
cleaner  a  r.d  more  productive  coding  but  also  to  easier  and 
more  flexible  testing.  The  advantages  with  respect  to 
debugging  ard  testing  show  up  in  several  ways.  Single 
modules  can  be  debugged  and  tested  independently  from  the 
other  modules  or  the  main  (driver)  program.  Furthermore,  if 
the  modules  are  snail  enough,  extensive  testing  generally 
assumed  as  impossible  with  the  exception  of  very  trivial 
programs,  can  become  manageable.  This  in  turn  leads  to  more 
reliable  programs.  If  all  modules  of  a  software  project  can 
be  tested  extensively,  a  highly  reliable  program  can  be 
produced.  iven  if  one  falls  short  of  this  goal  -  and  this 
happens  ir  most  cases  due  to  the  very  large  number  of 
possible  inputs  and  program  paths  -  the  final  program  will 
be  more  reliable  and  more  thoroughly  tested  than  a 
ncn^fflcdular  program.  The  possibility  of  testing  modules 
individually  provides  for  better  (more  eccncnical) 
allocation  of  testing  resources,  because  cne  does  not  have 
to  wait  until  the  whole  program  has  been  completed.  However, 
to  test  individual  modules,  special  test-routines  are  needed 
as  drivers  and  if  other  modules  must  interact,  dummy  modules 
must  be  created  if  tie  real  modules  are  not  yet  available  or 
not  yet  tested. 
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One  final  point  in  favour  of  modular  programming  has  to 
be  made:  Normally,  no  production  program  is  completed  unril 
the  day  when  it  is  no  longer  used,  i.e.  every  running 
production  program  has  to  be  maintained  and  adapted  to  new 
considerations  and  situations.  Because  of  the  simplicity  of 
the  overall  crgani2ation  of  modular  programs  this  software 
maintenance  is  alleviated  since  interactions  between  modules 
are  more  easily  understood;  hence,  the  effect  of  program 
changes  is  easier  to  identify.  Also  only  the  modules 
affected  ky  the  change  have  to  be  tested  (together  with  the 
main  program  and  interacting  modules) . 


fl.  SIBDClDfiir  PECGEAKMING 


Having  coded  a  program  in  the  above  described 
modularized  fashion;  there  is  still  room  for  improvement. 
Since  Dijkstra’s  famous  letter  to  the  editor  of  the 
Communicaticrs  of  the  ACM  in  which  he  proposed  to  eliminate 
GO-IO  statements  [5]^  the  concept  of  Structured  Programming 
has  evolved  and  led  to  further  simplification  of  the  coding 
process. 

Simplification  means  here  not  that  the  actual  code  is 
easier  to  write  -  although  this  might  be  the  case  toe  for  a 
programmer  who  is  familiar  with  the  concept  and  can  think  in 
these  terms  -  but  the  code  produced  and  the  control  secuence 
of  the  finished  program  is  simpler  than  in  a  non  structured 
pregram.  Ibis  simplification  has  been  theoretically 
demonstrated  ty  Boehmi  and  Jacepini  as  early  as  1966  [3]. 
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Although  there  are  as  many  interpretations  of  what 
Structured  Erogramming  is  as  there  are  authors  oc  this 
topic,  tte  following  features  are  essential  and  commcn  to 
this  concept: 

(1)  ICP-^ICWN  Desggn,  i.e.  the  design  starts  at  a  very 
general  level  and  proceeds  stepwise  to  the  specific 
and  detailed  tasks 

(2)  Modular  Design 

(3)  limited  possibilities  to  control  the  logic  flow  of 
the  program,  namely  only 

*  seguential 

♦  conditional:  IF  -  THEN  -  ELSE 

=*  iterative:  DO  -  WHILE 

statements  are  allowed. 

Whereas  the  so  called  block-structured  languages  like 
ALGOL  cr  EL/I  lend  themselves  to  this  form  of  coding 
(although  GC-IO  statements  are  provided'  by  the  language), 
even  in  ECBTEAN  the  implementation  of  seme  of  the  basic 
principles  of  Structured  Programming  is  possible  if  the 
programmer  concerned  with  a  structural  flew  cf  his  program 
cheeses  the  branching  caused  by  unavoidable  GC-TO 
statements  carefully. 

Eaker  [1]  shows  that  the  application  of  Structured 
Programming  combined  with  the  ’’Chief  Programmer  Team  Method" 
of  organising  a  software  project  [2]  can  bring  measurable 
improvements  in  software  development,  in  the  coding  as  well 
as  in  the  debugging  and  in  the  testing  stage.  Due  to  the 
fact, that  Structured  Programming  implies  Modular  Programming 
the  same  advantages  hold  here  too,  i.e.  the  software  is 
easier  tc  test  and  tp  maintain  after  release. 
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IV. 


IHJ  PROBLEM  CF  PBOGBAM  COMBIEXITY 


The  inpact  of  th£  programniing  technigues  descrihed  afcove 
cn  the  €CCECiric  development  of  reliable  and  maintainable 
software  is  directly  related  to  the  problem  of  program 
complexity.  There  is  so  far  no  generally  adopted  definition 
of  what  program  complexity  really  means.  The  definition  is 
dependent  or  the  context  in  which  one  wants  to  examine 
program  complexity.  Here  complexity  is  defined  as  structural 
properties  of  a  program  that  affect  the  ability  to  detect 
errors. 

Dnder  the  condition  that  the  structure  of  a  program  is 
described  by  a  directed  graph,  the  following  criteria  can  be 
used  to  measure  its  complexity: 

1.  Number  of  nodes 

2.  Number  of  arcs 

3.  Number  of  possible  paths  through  the  program 

4.  Number  of  source  statements 

5.  Averace  path  length  (source  statements  per  path,  arcs 
per  paths) 

6.  Beachability  index 

7.  Fullness  index  (ratio  of  actual  to  maximum  number  of 
arcs)  . 

Although  Mills  in  his  contribution  to  Eef.  8  generates  the 
idea  of  equating  program  complexity  with  the  difficulty  of 
understanding  a  prpgram  and  justifies  this  approach  with 
'‘.-■..'the  frustration  of  concocting  and  demolishing  more 
simple  minded  direct  ideas,  such  as  counts  of  branches,  data 
references,  etc.",  his  approach  does  not  help  to  get  a  real 
measurement  of  complexity  such  that  one  is  able  to  make  a 
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quantitative  stateae'Ut  how  complex  a  program  is.  It  seems 
that  the  important  point  is  to  relate  program  complexity  to 
the  problem  area  one  pursues.  The  analysis  of  NT DS-E cgrams 
has  given  insight  in  methods  to  measure  complexity  vith 
respect  to  problems  of  program  design  and  testing. 
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V. 


JBICR  detection  simulation  mod^ 


A.  GENEEAL 


A  Software  Error  Detection  Simulation  Model  was 
criginallj  developed  by  T.F.  Green  in  his  M.S.  Thesis  [7] 
and  subsequently  mpdified  by  professor  G.T.  Howard  of  the 
Naval  Postgraduate  School.  Written  in  FORTRAN  it  was 
designed  to  run  on  the  IBM  360/67  computer  of  the  Naval 
Postgraduate  School.  Originally  it  had  been  tested  against 
hypothetical  and  actual  programs.  It  was  shown  that 
sinulaticD  of  error  detection  was  feasible  and  that 
information  cculd  be  obtained  on  the  relationship  between 
error  detection  and  program  complexity.  However,  it  was 
necessary  to  perform  additional  model  feasibility  tests  by 
using  the  model  on  a  large  number  of  actual  programs.  In 
the  process  of  testing  some  of  the  original  features  had  to 
be  removed  and  provisions  had  to  be  made  for  cases  of 
program  behaviour  which  were  unexpected  at  the  time  of  the 
simulation  program  design.  A  detailed  description  of  the 
model  with  its  specific  assumptions  and  capabilities  is 
found  in  Eef.  10,  pg,-  IV-5  -  IV-39. 

E.  PRCGBAM  EEPRESENTATION 


The  prerequisite  for  the  use  of  the  simulation  mccel  is 
to  get  the  structure  of  a  program  that  has  to  be  tested  in 
the  form  of  a  directed  graph.  A  directed  graph  is  a 
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convenient  D-eacs  to  show  the  structure  of  prcgrams.  It  is 
suitable  for  showing  the  control  flow  in  a  program,  measures 
of  compleaitj  can  be  derived  from  this  kind  of 
representation.  In  addition,  the  *’ccntrcl  flow  graph"  as 
this  ccmpcsiticn  cf  structures  is  sometimes  called,  is  also 
very  useful  for  determining  the  execution  time  of  a 
structure  cn  a  machine.  This  representation  of  pregram 
structures  also  simplifies  the  representation  of  large  and 
ccnplex  programs  because  these  programs  can  be  broken  up  in 
logical  segirents  (modules,  procedures,  subroutines  etc.), 
and  the  segments  can  be  tested  separately  from  the  ether 
parts  cf  tie  program. 


C.  CUBRiKT  STATDS  0«F  THE  SIMUIATICN  PEOGB^H 


1. 

Input 

Variables 

The 

following  input 

variables  have  to 

be  used  for 

the  si 

mu laticn : 

a . 

MINPUI 

designates  the 

number  of  inputs 

withic  each 

replication. 

b. 

NOaODT 

is  the  number 

of 

replications 

(number  of 

paths)^  within  every 

repetiticE . 

c . 

NEEPEI 

is  the  number 

cf 

reseedings 

with  errors 

(repetitions)  . 

d. 

MEANLE 

designates  the 

mean  arc  length  if  the  arc 

lengths  are  selected 

at  random  by 

the  pregram 

and  are  not  read 

in . 

.  e. 

MEANEE 

designates  the  mean 

number  cf 

instructions 

between  errors. 

f . 

N 

is  the  number  cf 

nodes  within  the 

strucrure- 

g- 

Input 

fer  the  Adjacency 

Matrix  is  done  in 

a  shorthand 
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nctaticn: 

Per  €very  node  with  the  exception  of  the 
there  is  one  data  card  which  contains  i 
atcut  this  node  in  the  following 
Identification  of  the  node,  number  cf  arcs 
frcffi  this  node,  identification  numbers  of 
tc  which  the  arcs  go. 

h.  Input  for  the  matrix  of  arc  lengths 
siailar  to  that  for  the  adjacency  matrix: 
only  as  the  identifiers  for  receiving  node 
(identifier,  number  of  statements  on  this  a 
be  provided. 

i.  Input  tc  plant  errors  in  arcs  instead  cf  le 
pregram  seed  them  at  random:  Input  as  for 
arc  length,  but  the  number  of  errors  on  thi 
tc  he  specified  instead  of  the  number  cf  st 

j.  MCDT  specifies  the  desired  output: 

0  =  Summary  output 

1  =  Extensive  output  (NUMOOT  *  NHEPEI  <  25) 


2-  Incut  Eorm ats 


Ihe  input  formats  are  as  follows; 

First  data  card:  (615)  aiNPGT,  NUMODT,  NHBPE 

MEANES,  K. 

Secend  and  following  cards;  adjacency  matrix 
followed  hy  delimiter-card:  S9  in  columns  4  and  5. 

Input  cards  for  matrix  of  arc  length  (cptio 
7(I5,F5.C);  followed  by  delimiter-card:  99  in  col 
5.. 

Input  to  seed  errors  manually  (optional)  :  1615; 

delimiter-card:  99  in  columns  4  and  5. 


last  nodes 
nf or  ration 
seguence: 
emanating 
the  nodes 

(opt  ional) 
Instead , 
s  the  pair 
rc)  has  to 

ttinc  the 
matrix  of 
s  arc  has 
atements . 


T,  EEAJIIN, 

,  (1615); 

n al)  :  2l5 , 
umns  4  and 
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Last  data  caid  (output  specification):  15. 

Note  that  all  delimiter  cards  are  not  optional. 

lim itations 

Ihis  simulation  program  is  currently  restricted  to 
accomodate  a  maximum  number  of  30  nodes.  The  execution  time 
for  simpler  structures  (about  10  -  15  ncdes)  is  within  a 
five  minute  time  limit.  Larger  and  more  complex  structures 
with  more  nodes  and  possible  paths  through  the  structure 
reguire  a  30  minute  time  frame  for  the  execution  of  one 
simulated  input  in  100  replications  and  100  repetitions. 

An  extension  of  the  limits  of  the  program  to 
accomodate  larger  structures  seems  to  be  impractical  because 
of  the  fast  rise  of  memory  space  and  execution  time 
reguired . 

ilPSlU  Listing 

A  listing  of  the  current  error  detection  simulation 
program  as  it  was  used  for  the  analysis  of  the  MTDS-prcgrams 
is  found  in  Appendix  A. 
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VI.  ANALYSIS  OF  NTDS  FROG  FANS 


A.  GENEFAl 

In  order  to  dempnstrate  the  practicality  of  program 
analysis  using  the  Error  Detection  Simulation  Model,  Naval 
Tactical  Data  Systems  Programs  have  been  analyzed  by 

1.  describing  the  structure  by  converting  the  programs 
into  the  form  of  directed  graphs 

2.  running  these  structures  on  the  error  detection 

simulation  model  and 

3.  evaluating  the  simulation  results  with  respect  to 
measures  of  program  complexity. 


B.  DESIGN  Cl  NTDS  EflCGEAMS 


^cdular  Design 

Ihe  design  of  NTDS  programs  is  characterized  by 
Modular  Frccramuing,  both  in  general  and  in  detail,  and  the 
modular  design  is  a  characteristic  of  the  hardware  as  well. 
Also  the  actual  implementation  of  every  NTDS  installation 
consists  of  hardware  and  software  building  blocks  that  are 
composed  to  fit  exactly  the  need  of  each  installation. 

Although  NTDS  programs  are  really  programmed  in  a 
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mcdular  fashicn,  tJie  term  ''module”  does  net  have  ths  same 
meaning  as  usual.  Mpdule  usually  refers  to  basic  building 
blocks  that  are  farts  of  the  program,  whereas  NTDS  programs 
are  composed  of  subsystems.  The  NTDS-”Modul€s”  in  turn  are 
divided  up  in  parts  which  correspond  to  the 
”mcdule”-def initicn  of  Modular  Programming.  In  NTDS 
terminology  these  parts  are  called  procedures.  NTDS  modules 
perform  complex  tasks  such  as  tracking,  display  etc.  They 
certain  a  ledium  to  large  number  of  dependent  prccecures. 
These  preoedures  perform  the  basic  functions  intended  in 
Modular  fregramming  such  as  checking  track  properties. 
Throughout  this  discussion,  "module”  is  used  as  in  the  NTDS 
system,  namely  as  a  complete  subsystem  for  performing 
complex  tasks. 

The  mcdular  approach  is  imbedded  in  a  stringent 
hierarchical  system  which  is  controlled  by  the  priorities  of 
the  tasks  to  be  performed.  The  levels  of  hierarchy  are 
applied  to  the  modules  in  such  a  way  that  only  major 
subprograms  which  are  designed  to  execute  distinctive  tasks 
can  communicate  with  each  other,  whereas  the  procedures 
within  the  modules  can  only  communicate  according  to  xhe 
level  of  hierarchy  they  belong  to,  with  the  exception  of 
calls  to  certain  system  rourines. 

2 .  CS-^  language 


The  NTDS  programs  are  written  using  the  CS- 1  high 
level  language  compiler  [6].  This  language  has  the  advantage 
that  it  is  well  suited  to  the  application  area,  namely 
tactical  programs  which  run  under  severe  constraints 
regarding  time  and  memory  space  availability.  Tables  are 
searched  in  a  very  effective  way,  and  another  interesting 
feature  is  that  assembly  code  can  be  interspersed  within  the 
high  level  code  of  the  program.  This  fact  gives  the 
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frcgrammer  a  powerful  means  for  controlling  the  hardware 
which  in  tnrc  facilitates  the  production  of  effective  code. 


C.  LIBECIED  GBAEH  CCNSTEUCTICN 


In  order  to  ottain  the  desired  statistics  and  to  analyze 
the  data-  and  control  flow  of  a  single  NIES-progran ,  the 
following  method  has  been  developed  and  used: 

1.  Cne  oomplete  module  from  an  existing  and  currently 
operating  KIDS  program  has  been  put  into  the  form  of 
a  directed  graph.  The  module  has  heen  deccapcsed 
intc  the  procedures  it  contains,  and  every  procedure 
is  treated  separately.  Due  to  the  modular  design 
tbcughcut  the  program,  no  logical  difficulties  arise 
here,  because  every  procedure  has  only  one  entrance 
and  cne  exit  point,  i.e.  the  interface  for 
interacting  procedures  within  the  module  is  uniquely 
defined.  For  each  procedure  the  directed  graph  and 
the  adjacency  matrix  have  been  constructed.  As 
quantitative  measurements  the  number  of  nodes,  arcs, 
paths,  loops,  source  statements,  machine 
instructions,  source  statements  per  arc,  and  machine 
instructions  per  arc  have  been  compiled. 

2.  The  same  work  was  done  for  randomly  selected 
procedures  from  one  other  important  module  of  the 
same  prcgran  in  order  to  obtain  comparative  results 
and  to  relate  the  reported  number  of  errors  to  the 
different  modules. 
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3.  Ecr  the  construction  of  the  directed  graphs  and  the 
gathering  of  the  several  statistics  the  following 
assumptions  have  been  made: 

a.  Nodes  are  associated  with 


(.1)  Procedure  entrance  and  exits 

(2)  IF-statements  (decision  points) 

(3)  Points  where  paths  merge 

(Si)  Procedure  calls  within  the  module 

(5)  Beginning  and  ending  cf  loops 

t.  All  nodes  within  the  module  are  distinct. 
They  have  individually  assigned  numbers  (some 
nodes  are  indicated  as  "dumity'*  nodes)  ,  and 
they  are  counted  only  once,  namely  ir  the 
procedure  they  belong  to. 


Entrance  and  exit  nodes  of  a  called  procedure 
are  regarded  as  "transient"  nodes  within  the 
calling  procedure,  and  one  "transient  arc" 
connects  both  transient  nodes.  This 
transient  arc  represents  all  the  arcs  inside 
the  called  procedure.  The  transient  arcs  are 
indicated  in  the  drawings  by  a  dashed  line. 
Transient  nodes  have  either  the  number  cf  the 
entrance  node  of  the  called  procedure  or  they 
are  denoted  by  letters  tc  distinguish  them 
frcm  the  original  nodes  of  the  corresponding 
procedure. 


d.  The  langth  of  every  arc  is  indicated  as  the 
number  of  source  statements  cr  the  number  of 
machine  instructions  respectively.  Ir  the 
analysis  the  number  cf  source  statements  has 
been  used  because  programs  are  normally 
written  in  a  high  level  language  and  this  is 
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the  point  where  errors  are  introduced  into 
the  program. 

4.  Normally,  the  numbers  of  both  source  statements  and 
machine  instructions  have  been  counted  in  the  arc 
where  the  statements  appear.  However,  because 

Il-statements  and  procedure  calls  result  in 

branching,  they  have  been  counted  in  the  arc  leading 
tc  the  corresponding  node.  Whereas  for  the  counting 
of  machine  instructions,  it  would  be  possible  in  the 
case  cf  an  IF-statement  to  split  the  instruction 
sequence  according  tc  the  arcs  emanating  froa  the 
decision  point,  this  is  not  feasible  for  the  source 
statement  which  contains  the  elements  of  both  arcs 
eaanating  from  it;  it  cannot  be  split. 

The  structures  obtained  from  bcth  modules  anc  the 
compiled  statistics  are  found  in  Appendix  E.  The  following 
figure  shews  how  tc  read  the  structure  diagrams: 


Figure  2  -  EXAMPLE  OF  FROGHAM  SIEUCT05F 
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c. 


EBSOB  DEIECTION  SIMOLATICN  ON  THESE  STBDCTUBES 


The  structures  which  were  converted  into  directed  craphs 
for  Module  One  were  screened  to  determine  their  suitability 
for  errcr  detection  simulation.  It  would  have  teen 
desirable  to  select  a  random  sample  of  the  structures. 
However,  it  was  necessary  to  choose  structures  which  wculd 
not  require  excessive  amounts  of  memory  space  and  CPU  time 
during  the  simulation.  In  addition,  the  structures  were  to 
have  at  least  two  or  more  paths.  In  the  case  of  Module  Two 
it  was  leasitle  tp  use  a  random  sample  because  a  high 
percentage  of  the  structures  fell  within  the  memory  space 
and  the  CPU  time  limitations  of  the  model. 

The  input  data  for  the  simulation  were  taken  from  the 
actual  programs,  including  the  number  cf  source  statements 
for  every  arc.  The  recorded  number  of  errors  per  module  was 
used  to  calculate  the  mean  number  of  instructions  between 
errors,  which  is  used  for  seeding  errors  in  the  simulation 
model.  Seeding  the  errors  was  done  randomly  by  the 
simulation  program.  However,  it  was  provided  thar  no  errors 
were  seeded  at  arcs  containing  zero  instructions  (control 
arcs)  . 

The  simulation  was  run  with  one  input,  100  replications 
and  100  repetitions  (reseedings) ,  and  the  average  number  of 
errors  found  by  one  input  was  obtained.  although  seme  of 
the  structures  were  small,  and  a  higher  number  of 
repetitions  and  replications  could  have  been  run,  the  same 
simulation  parameters  were  used  for  each  structure  in  order 
to  obtain  ccaparable  results. 
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E. 


RESUIIS  CE  THE  A^NALYSIS 


Erom  th€  average  of  errors  found  by  one  input  in  each 
procedure  the  average  percentage  of  errors  fcund  against  the 
errors  expected  within  the  procedure  was  obtained.  These 
results  were  plotted  against  various  complexity  measures/ 
e.g.  the  cumber  of  paths.  Although  the  results  varied 
scmewhat  between  tie  modules,  it  was  possible  to  establish 
relationships  between  structural  properties  and  error 
detection  capabilities. 

The  differences  in  results  between  modules  can  he  traced 
to  several  factors: 

1,  Different  sample  sizes: 

From  Module  One  32  procedures  were  used,  16 
procedures  were  randomly  selected  frcm  Module  Two. 

2.  The  different  size  of  the  modules: 

Module  One  had  97,  and  Module  Two  had  155  procedures, 

3.  Differences  in  program  design  and  programming  style: 
Module  Two  was  modularized  to  a  much  larger  extent 
than  Module  One.  It  was  hard  to  find  a  sufficient 
number  of  paths  within  randomly  selected  procedures 
of  Module  T wo. 

4,  Different  number  of  reported  errors: 

Although  Module  Two  was  1.6  times  larger  than  Module 
One,  it  had  only  about  two-thirds  the  number  of 
errors . 

The  following  diagrams  show  the  percentage  of  average 
errors  fcund  against  the  expected  number  of  errors  for  the 
structures  of  both  modules. 
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•  Percent  of  expected  errors  Percent  of  expected  errors 


1 


Ccdule  Cne 


100  1 


^  To  T5  2^  ?5  3^ 

nodes 

figure  3  -  PERCENTAGE  ERRORS  FCONE  VS.  NODES 
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Percent  of  expected  errors  Percent  of  expected  errors 


100  -1 


^  To  T5  2^  S  3^  35 

paths 

figure  5  -  PERCENTAGE  ERRORS  FODNE  VS.  PATHS 


inn 


figure  6  -  PERCENTAGE  ERRORS  FOUND  VS.  SOURCE  STATEMENTS 


30 


Percent  of  expected  errors  Percent  of  expected  errors 


2 


ficdule  Two 


100 


I  i  I  1  I  I  t 

5  10  15  20  25  30  35 

nodes 

figure  7  -  PEBCENTAGZ  EEBOES  FOONE  VS.  NODES 
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Percent  of  expected  errors  Percent  of  expected  errors 


100, 


50 


5  10  15  20  25  30  35 

paths 

Figure  9  -  PERCENTAGE  ERRORS  FODNC  VS.  PATHS 


Figure  1C 


PERCENTAGE  ERRORS 


FOUND  VS. 


SOURCE  STATEHETS 
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I 


The 

curves  shown  represent 

ex 

pcnential 

approximations 

to  the  datapoints  according 

to 

zhe 

formula 

y=a*€** (-b*x) 

which  was  found  to  represent 

the 

rel 

ati enship 

best.  A  least 

Square  fit  was  used. 

All  diagrams  show  seme  relationship  between  error 
detection  and  complexity.  Module  One  with  its  larger  sample 
size  shews  this  relationship  more  than  Module  Two  fer  the 
number  of  paths.  This  seems  logical  because  a  large  number 
cf  paths  reduces  the  ability  to  detect  errors  in  a  program. 
It  appears  that  the  number  of  paths  could  be  used  as  a 
measure  cf  program  complexity  for  design  and  testing 
purposes . 

In  order  to  rank  the  approximations,  a  squared  error 
factor  has  teen  computed  for  every  complexity  measure  as 
follows : 


Error 


Mod. 

Nodes 

7337 

Arcs 

6841 

Eaths 

4995 

£. stmts. 

6575 

This  computation 

shojjs  that 

paths  is  the  best  approximat 
method  used,  incther  interesting 
approximated  relationship  betwee 
and  the  number  cf  source  stateme 


Factor 
Mod.  2 

4430 

3933 

4666 

1808 
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of 
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t  he 
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cf  er 
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module 
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VII.  DSE  OF  THE  RESULTS 


A.  AILS  FCE  SOFTWARE  DEVELOEflENT 

This  methcd  cf  program  analysis  provides  the  so 
manager  vith  information  for  selecting  structures  easi 
the  design  process.  He  can  choose  the  least  c 
structure  shich  Mill  satisfy  project  require 
Furthermore,  after  a  project  has  been  coded  and  is  d 
testing,  he  can  make  realistic  assessments  ccncernin 
effort  which  will  be  needed  for  program  testi 
considering  factors  such  as 

1.  expected  complexity  of  the  project 

2.  choice  cf  the  programming  techniques  used 

3.  organixatioE  and  experience  of  the  programming 

4.  available  manp^ower  and  computer  time  for  t 
pur  poses. 

B.  FDTDBE  WCRK 

The  analysis  done  on  the  NIDS  programs  and  the  r 
obtained  fcr  the  measurement  of  program  comp 

represents  a  modest  contribution  to  the  field  of  so 
engineering.  But  being  far  frcm  complete  or  exhaust! 
following  steps  should  be  taken  in  order  to 

additional  validation  of  the  analysis  process. 


f tware 
ly  in 
cmplex 
nents. 
ue  for 
c  the 
ng  by 


team 

esting 


esults 
lexity 
f tware 
V e  the 
obtain 
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1 .  lurth€r  sva luation  of  NTDS-Modules 

Additional  NIDS  modules  should  be  evaluated  in  order 
to  obtain  larger  sample  sizes.  It  is  realized  that  the 
evaluation  process  for  the  important  modules  is  verj  time 
consuming.  However,  the  more  important  modules  are  used 
more  freguentlj  and  will,  in  most  cases,  have  a  longer  error 
history,  which  will  provide  valuable  data  for  comparison 
with  simulation  results. 

2*  Hvaluaticn  of  structured  programs 

It  would  be  of  interest  in  this  respect  to  compare 
the  evaluation  of  the  NTDS-prccedures  with  procedures  that 
perform  the  same  functions  but  are  rewritten  and  converted 
into  a  structured  programmed  form.  It  is  expected  that  the 
structured  programs  would  perform  better  with  respect  to 
error  detection. 
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VIII.  SUMMABY  AND  CONCIUSICNS 


A  metiiod  to  djEfine  and  analyze  prcgran  structures  has 
keen  presented.  All  measureinents  obtained  were  based  cn  the 
descripticn  cf  the  program  structure  in  the  fcris  cf  a 
directed  crajh  and  the  use  of  the  error  detection  simulation 
mcdel.  Ihis  methcd  has  been  used  to  analyze  the  procedures 
frcm  two  KIES  modules.  It  was  beyond  the  scope  cf  this 
effort  tc  obtain  ccmparative  results  between  this  experiment 
and  the  actual  error  history  cf  the  programs.  However,  it 
was  possible  tc  obtain  an  initial  guantitative  assessmenr  of 
measures  cf  complexity. 

By  using  this  method  to  check  program  structures  in  the 
design  phases  it  should  be  possible  to  produce  programs  with 
structures  that  are  less  complex  and  therefore  easier  and 
more  economical  to  test  and  maintain.  Also  the  methcd  cculd 
he  used  during  the  test  phase  as  a  means  cf  assigning  test 
resources. 
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APPENDIX  A 


EEROB  DETECTION  SIMOLAIICN  PSCGEAM 


The  p 
Software 
analysis 
May  1976 
he  regard 


rogian  listed  on  the  following  pages  s 
Error  Detsction  Simulation  Progran  as  us 
of  the  NIBS  procedures  in  the  version  cu 
.  Although  carefully  tested  the  program  s 
ed  as  a  final  release. 
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8Ce  CCMINUE 


APEENDIX  B 


IIST  CF  EVALUATED  PROGRAil  STRUCTURES 


This  list  gives  all  the  statistical  data  gathered  frcic 
the  ccnversicn  of  the  procedures  of  the  NTES  modules  into 
the  fori  of  directed  graphs.  The  abbreviations  read  as 
fellows : 


PUR 

R 

A 

P 

L 

Ss 

Mi 


Procedure 
Nuiber  cf 
Nuttber  of 
Nunber  of 
Nuiber  cf 
Number  of 
Number  of 


number  within  the  module 

nodes  (including  transient  nodes) 

arcs  (including  transient  arcs) 

paths 

loops 

source  statements 
machine  instructions 


SA  Sccrce  stmts. /arc 

MA  Machine  instr./arc 
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4 

4 

2 

0 

7 

14 
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47 

56 
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3 

44 
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12 

12 

2 

1 

10 

16 
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26 

27 

1 

0 

25 

44 

0.93 

1.63 

8 

8 

2 

1 

8 

30 

1  .  00 

3.75 

49 

61 

12 

0 

53 

90 

0.87 
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6 

7 
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0 

8 

22 
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3.14 
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7 
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23 
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25 

31 
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23 
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44 

57 

1  1 

0 

55 
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8 

9 

3 

0 

7 

16 
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6 

5 

1 

0 

8 

18 

1  .  60 

3.  60 

13 

14 

3 

0 

8 

20 

0.57 

1.43 

91 

120 

25 

0 

93 

191 

0.78 

1.59 
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N 

A 

P 

65 

33 

43 

219 

66 

18 

23 

13 

67 

2  1 

22 

6 

89 

51 

65 

14 

SO 

7 

10 

5 

91 

22 

30 

9 

S2 

5 

6 

3 

S3 

25 

34 

12 

S4 

7 

10 

5 

S5 

18 

27 

10 

S6 

45 

52 

35 

S7 

136 

211 

3972 

SS 

MI 

S/A 

M/A 

32 

93 

0.74 

2.16 

22 

56 

0.96 

2.  43 

25 

81 

0.93 

1.37 

54 

107 

0.83 

1.65 

8 

22 

0 .80 

2.20 

14 

47 

0. 47 

1. 57 

9 

28 

1  .  50 

4.67 

34 
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1.00 
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12 

37 

1  ,  20 

3.70 

19 

58 

0.70 

2.  15 

40 

93 

0.77 

1. 79 

99 

162 

0. 47 

0.77 

L 

0 

0 

0 

0 

0 

0 

0 

0 

2 

0 

1 
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2. 

ENE 

ecdul€ 

M 

T^o 

A 

P 

3 

2 

1 

1 

5 

6 

5 

1 
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2 

1 

1 
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2 

1 

1 

15 

1 1 

13 

5 

23 

1C 

1 1 

3 

40 
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27 

5 

4  1 

10 

14 

12 
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25 

37 

36 

47 

24 

34 

36 
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16 

21 

14 

54 

6 

5 

1 
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8 

8 

2 

59 

6 

5 

1 

65 

6 

5 

1 

69 

4 

4 

2 

73 

18 

22 

10 

79 

13 

14 

5 

82 

23 

24 

2 

86 

30 

34 

6 

90 

13 

18 

8 

99 

25 

30 

10 
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6 

5 

1 

114 

4 

4 

2 

121 

6 

5 

1 
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18 

21 

6 
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37 

46 

13 
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9 

9 

2 
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13 

17 

1 1 

149 

18 

25 

9 

SS 
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S/A 

M/A 

7 

7 

7.00 
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3 

4 

0.60 

0.80 

3 

4 
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9 

23 
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12 

31 
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12 

33 
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14 

30 
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13 

42 
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34 

95 
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34 

85 

1 . 00 
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17 

55 
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34 

97 

1 . 55 

4.41 
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34 
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35 

64 

1 .  46 

2.  67 

33 

86 
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23 

71 
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23 
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12 
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37 
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53 
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APPENDIX  C 


DIBECTED  GBAPHS 


CD  the  fcllcwing  fages  the  structures  of  all  the 
procedures  are  listed  that  were  used  as  input  data  for  the 
Error  Detection  Simulation  Model.  In  addition  tc  the 
complexity  measures  used  also  listed  are  the  results 
ohtained  frcm  the  simulation,  the  average  number  of  errors 
found  with  1  input,  ICO  replications  and  100  repetitions, 
and  the  percentage  of  expected  errors  detected. 

Differently  tc  the  sample  structure  shown  in  Fig. 
2,  the  number  cf  statements  is  indicated  in  the  following 
graphs  only  fcr  arcs  with  ncnzerc  instructicns. 


Ihe  count  for  the  number  of  nodes  and  the  number  of 
arcs  includes  the  transient  nodes  (designated  by  letters) 
and  the  transient  arcs  (dashed  lines)  because  they  must  be 
included  into  the  inpufs  for  the  Error  Detection  Simulation 
Program . 
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aodule:  1 


Procedure  No.:  2 


Nuffter  of  nodes: 

14 

Numter  of  arcs: 

23 

Nuiiter  of  paths: 

26 

Nun her  of  sctrce  stmts.: 

37 

Average  errcr  found: 

0.3144 

Percentage  ericrs  found: 

17.84 
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module;  1 


Procedure  No. ;  8 


Nuiifcer  of  nodes: 

13 

Nuoiter  of  arcs: 

14 

Nuater  of  paths: 

3 

Nuirbet  of  sccice  stmts.: 

10 

Average  error  found: 

0.2523 

Percentage  errors  found: 

52.98 
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Module 


1 


Procedure  No . :  11 


Nuiiher  of  nodes:  6 

Nutther  of  arcs:  8 

Nuiiter  of  patis:  5 

Nuiiiet  cf  sctrce  stmts.:  8 

Average  error  found:  0.1974 

Percentage  errors  found;  51.82 
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Med  ule 


1 


Procedure  No.:  14 


NuEter  of  nodes: 

Nuttfcer  of  arcs: 

Nueber  of  paths: 

Nuttter  cf  source  stmts.: 
Average  error  found: 
Percentage  errors  found: 


6 

7 

4 

9 

0.2536 

60.34 
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Module:  1 


Procedure  No. :  19 


Nuater  of  nodes: 

19 

Nuniter  of  arcs: 

26 

Nuaber  of  paths: 

7 

Number  of  source  stmts.: 

45 

Average  error  found: 

0.2885 

Percentage  errors  found: 

27.54 
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Module:  1 


Procedure  No.: 


22 


Nuifcer  of  codes: 

25 

Hunter  of  arcs: 

30 

Nuaher  of  paths: 

1 1 

Hunter  of  sccrce  stmts.: 

30 

Average  error  found: 

0.4105 

Percentage  errors  found: 

28.74 
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Mcdule:  1 


Procedure  No. 


Nuater  of  nodes;  12 

Nuiiter  of  arcs;  12 

Nuater  of  paths;  2 

Nuater  cf  sctrce  stmts.;  8 

Average  error  found;  0.2324 

Percentage  errors  found:  61.01 


Module:  1  Procedure  No.: 


4/16 


Nuafcer  of  nodes: 

17 

Number  of  arcs: 

19 

Number  of  paths: 

4 

Number  of  socrce  stmts.: 

32 

Average  error  found: 

0.6400 

Eercentace  errors  found: 

42.00 

28 
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Module 


1 


Procedure  No 


29 


Nuober  of  nodes:  28 

Sumber  of  arcs:  32 

Nuiiber  of  paths:  9 

Nuaber  of  source  stmts.:  47 

Average  error  found:  1.3946 

Percentage  errors  found:  62.31 
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Module 


1 


Procedure  No.:  30 


Nuafcer  of  nodes: 

7 

Nuflifcer  of  arcs: 

10 

Nuiiter  of  paths: 

5 

Nuafcer  cf  scrice  stmts.: 

10 

Average  error  found: 

0.  1649 

Percentage  errors  found: 

34.63 
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Module 


1 


Procedure  Ho. :  3^ 


Nuffker  of  nodes:  16 

Hunter  of  arcs;  17 

Hunter  of  patts:  3 

Hunter  of  source  stmts.;  5 

Average  error  found;  0,5465 

Percentage  errors  found:  76.51 
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Module 


1 


Procedure  No.  : 


35 


Nuater  of  nodes; 

14 

Number  of  arcs: 

17 

Number  of  paths; 

3 

Number  of  source  stmts.: 

14 

Average  error  found: 

0.3576 

Percentage  errors  found; 

53.64 
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ilcdule:  1 


Procedure  Mo. :  36 


Hunter  of  ncces:  21 

Hunter  of  arcs:  26 

Hunter  of  paths:  3 

Hunter  of  sccrce  stmts.:  31 

Average  error  found:  0-5203 

Percentage  errors  found:  35.25 
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Module 


1 


Procedure  No 


39 


1/1 


Nuater  of 

ncdes: 

17 

Numter  of 

a  res : 

25 

Nunher  of 

paths: 

10 

Number  of 

serree  stmts.: 

17 

Average  error  found: 

0-2637 

Percentage 

errors  found: 

32.57 
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Module 


1 


Procedure  Ho.:  44 


Nuirfcer  of  nodes:  27 

Nuffiber  of  arcs;  30 

Nuiiter  of  paths:  7 

Nunber  of  source  stmts.:  21 

Average  error  found:  0,3554 

Percentage  errors  found:  35.54 
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Module;  1 


Procedure  No 


47 


Nuaher  of  nodes:  19 

Number  cf  arcs;  20 

Number  of  pa ths:  4 

Number  cf  source  stmts.:  12 

Average  error  found:  0.4231 

Percentage  errors  found:  74.04 
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Module;  1 


Procedure  No .  :  48 


Nuirter  of  nodes: 

23 

Nuirter  of  arcs: 

26 

Nuiiter  of  paths: 

7 

Number  of  source  stmts.: 

13 

Average  error  found: 

0.3287 

Percentage  errors  found: 

53.10 
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Mcdule:  1 


Procedure  Ilo . :  49 


Nuirter  of  nodes; 

15 

Number  of  arcs; 

18 

Number  of  paths; 

7 

Number  cf  source  stmts.; 

19 

Average  error  found: 

0.2217 

Percentage  errors  found: 

24.50 
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Module 


1 


Procedure  No . : 


53 


Nunher  of  nodes: 

1 1 

Nurher  of  arcs: 

18 

Nuffter  of  paths: 

9 

NuiEfcer  of  source  stmts.: 

1 1 

Average  error  found: 

0.1876 

Percentage  errors  found: 

35.81 
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Kcdule:  1 


Procedure  No. :  57 


Nunher  of  ncces:  30 

Nuaber  of  arcs:  36 

Nuater  of  paths:  14 

Number  of  sctrce  stmts.:  26 

Average  error  found:  0,2910 

Percentage  eircrs  found:  23.50 
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Module: 


1 


Procedure  No.:  60 


Nuater  of 

nodes: 

28 

Number  c£ 

arcs : 

37 

Nunfcer  of 

paths: 

18 

Number  cf 

sctrce  stmts.: 

24 

Average  errcr  found: 

0.3336 

Percentage 

eircrs  found: 

29.19 
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Procedure  No, 


75 


Nuiter  of  arcs:  28 

Nunher  of  paths:  8 

Number  cf  source  stmts.:  20 

Average  errcr  found:  0-5433 

Percentage  errors  found:  57.05 
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Module:  1 


Procedure  No 


76 


Nuiiber  of  nodes; 

15 

Number  of  arcs: 

19 

Number  of  paths: 

5 

Number  of  sccrce  stmts.: 

20 

Average  errcr  found: 

0.3893 

Percentage  errors  found: 

40.88 

76 


Module:  1 


Procedure  Ho . :  11 


Hunter  of  nodes: 

17 

Hunter  of  arcs: 

20 

Hunter  of  paths: 

9 

Humber  of  sctice  stmts.: 

10 

Average  error  fouxud: 

0.2425 

Percentage  errors  found: 

50.93 

11 


Module 


1 


Procedure  No. :  79 


Nuater  of  nodes; 

25 

Number  of  arcs: 

31 

Number  of  paths: 

3 

Number  cf  sccrce  stmts.: 

23 

Average  errcr  found: 

0.3628 

Percentage  eircrs  found: 

33.13 

78 


Hcdule:  1 


Procedure  No. :  81 


Nuiiiter  of  nodes:  8 

Nutcber  of  arcs;  9 

Nuiiter  of  paths:  3 

Nuffher  cf  scvrce  stmts.;  7 

Average  error  found:  0.1449 

Percentage  errors  found;  43.47 
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Module:  1 


Procedure  No.:  86 


Nuater  of  nodes: 

18 

Nuniter  of  arcs: 

23 

Nunter  of  paths: 

13 

Nuoifcer  of  source  stmts.: 

22 

Average  errcr  found: 

0.3370 

Percentage  errors  found: 

32. 17 

80 


Module 


1 


Procedure  No. ;  87 


Nuiter  of  ncces; 

21 

Number  of  arcs: 

22 

Number  of  patts: 

6 

Number  cf  scuice  stmts.: 

25 

Average  errcr  found: 

0. 

5029 

Percentace  eircrs  found: 

42 

.24 

81 


Module:  1 


Procedure  No.:  91 


Nuirter  of  nodes: 

22 

Numter  of  arcs: 

30 

Nuiiter  of  patis: 

9 

Nuiter  of  sccrce  stmts.: 

14 

Average  errci  found: 

0.  1438 

Percentage  errors  found: 

21.57 
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Module:  1 


Procedure  No. : 


92 


Nuaber  of  ccces;  5 

Number  of  arcs:  6 

Number  of  paths:  3 

Nuaber  of  sctice  stmts.:  9 

Average  error  found:  0.1837 

Percentage  errors  found:  42.86 
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Module 


1 


Procedure  No 


93 


Nuaher  of  nodes:  25 

Nuniher  of  arcs;  34 

Nuiiter  of  paths:  12 

Nujcter  of  source  stmts.:  34 

Average  error  found:  0.3972 

Percentage  errors  found:  24.53 
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Module 


1 


Procedure  No. : 


95 


Nuaher  of  nodes: 

18 

Hunter  of  arcs: 

27 

Nunher  of  paths: 

10 

Nuaiter  of  source  stmts.: 

19 

Average  error  found: 

0.  1822 

Percentage  errors  found: 

20.14 
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Hcdule:  2 


Procedure  No . :  15 


Nuafcer  of  ncces: 

1 1 

Nuiiter  of  arcs: 

13 

Nuirber  of  paths: 

5 

Nuttfcer  of  sccrce  stmts.: 

12 

Average  error  found: 

0. 

0836 

Percentage  errors  found: 

35 

.55 

86 


acdule;  2 


Procedure  No 


23 


Nuiiter  of  ncces: 

10 

Nuttber  of  arcs; 

1 1 

Nuiber  of  paths: 

3 

Nuiber  cf  sccrce  stmts.; 

12 

Average  errci  found: 

0. 

1592 

Percentage  errors  found: 

67 

.66 

87 


Bed  ule 


Procedure  No.:  40 


Nuafcer  of  ncces: 

22 

Nuttber  of  arcs: 

27 

Nuater  of  paths: 

5 

Nuiiher  of  sccice  stmts.: 

14 

Average  errer  found: 

0, 2018 

Percentage  errors  found: 

73.51 

88 


Med  ule 


Procedure  No 


4  1 


Nuaber  of  nodes: 

10 

Nuiiber  of  arcs: 

14 

Nunber  of  paths: 

12 

Nuaber  cf  setree  stmts.: 

13 

Average  error  found: 

0. 

1554 

Percentage  errors  found: 

60 

.96 

89 


Module : 


Procedure  No. ;  46 


Nuiiter  of  paths;  36 

Nunter  of  source  stmts.:  34 

Average  error  found:  0.1657 

Percentage  errors  found:  24.86 
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Module: 


Procedure  No. :  47 


Nunfcer  of  ncces: 

24 

Number  of  arcs: 

34 

Number  of  paths: 

36 

Number  of  sccxce  stmts.: 

34 

Average  errcr  found: 

0. 

1163 

Percentage  eircrs  found: 

17 

.45 
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Module 


2 


Procedure  Mo. :  48 


Nuiiber  of  nodes: 

16 

Number  of  arcs: 

21 

Number  of  paths: 

14 

Number  of  soutoe  stm-ts.: 

17 

Average  errcr  found: 

0. 

1580 

Feroentace  errors  found: 

47 

.40 

92 


Med  ule: 


2  Procedure  No. : 


Nuuter  of  ncces:  18 

Hunter  of  arcs;  22 

Nuirter  of  paths:  10 

Hunter  cf  source  stm.ts.;  34 

Average  error  found;  0,1885 

Percentage  errors  found:  28.28 
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93 


Hcdule: 


Procedure  No.:  7S 


Huifcer  of  nodes: 

13 

Nuiter  of  arcs: 

14 

Nuater  of  paths; 

5 

Nuniher  cf  sctice  stats.: 

1 1 

Average  errci  found: 

0. 

1379 

Percentage  errors  found: 

63 

.94 

94 


Module: 


Procedure  Nc .  :  82 


Nuiter  cf  nodes; 

23 

Nuiifcer  cf  arcs: 

24 

Nunfcer  of  paths; 

2 

Nuiher  cf  sctrce  stm.ts.; 

35 

Average  errcr  found; 

0. 

1 130 

Percentage  errors  found: 

16 

.47 

95 


Module ; 


Procedure  No.:  86 


Nuiiter  of  ncdes: 

30 

Nuirher  cf  arcs: 

34 

Hunter  of  paths: 

6 

NuEter  cf  sccice  stmts.: 

33 

Average  errci  found: 

0. 

2042 

Percentage  eircrs  found: 

31 

.56 

96 


Module: 


Procedure  No . :  90 


1/1 


Nunfcer  of  ncces: 

13 

Numher  cf  arcs: 

18 

NuEher  of  paths: 

8 

Nuafcer  cf  source  stmts-: 

23 

Average  error  found: 

0. 

0  95  8 

Percentage  errors  found: 

21 

.24 
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Module: 


Procedure  No.:  99 


Nuiiter  of  nodes: 

25 

Nunter  of  arcs: 

30 

Nunier  of  paths: 

10 

Nuiiber  cf  sccrce  stm.ts.: 

23 

Average  error  found: 

0. 

1513 

Percentage  errors  found: 

33 

.55 
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Module:  2 


Procedure  No.:  122 


Nuater  of  nodes: 

18 

Nufflfcer  of  arcs: 

21 

Nuirter  of  paths: 

6 

Nuiiher  of  sctrce  stats.: 

20 

Average  error  found: 

0. 

1686 

Percentace  errors  found: 

42 

.99 
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Module 


Procedure  No.:  157 


Nuiiter  of  nodes: 

13 

Number  of  arcs; 

17 

Number  of  paths: 

1 1 

Number  of  source  stmts.: 

23 

Average  error  found: 

0. 

1178 

Percentage  errors  found: 

26 

.12 
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Module; 


Procedure  No.:  149 


Nuiher  of 

nodes: 

18 

Nuttfcer  cf 

arcs: 

25 

Nuater  of 

paths: 

9 

Nunifcer  cf 

sctrce  stmts.: 

35 

Average  error  found: 

0.2357 

Percenta  ce 

errors  found: 

34.34 
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